Making Small Language Models Better Multi-task Learners with Mixture-of-Task-Adapters

Recently, Large Language Models (LLMs) have achieved amazing zero-shot learning performance over a variety of Natural Language Processing (NLP) tasks, especially for text generative tasks. Yet, the large size of LLMs often leads to the high computational cost of model training and online deployment. In our work, we present ALTER, a system that effectively builds the multi-tAsk Learners with mixTure-of-task-adaptERs upon small language models (with <1B parameters) to address multiple NLP tasks simultaneously, capturing the commonalities and differences between tasks, in order to support domain-specific applications. Specifically, in ALTER, we propose the Mixture-of-Task-Adapters (MTA) module as an extension to the transformer architecture for the underlying model to capture the intra-task and inter-task knowledge. A two-stage training method is further proposed to optimize the collaboration between adapters at a small computational cost. Experimental results over a mixture of NLP tasks show that our proposed MTA architecture and the two-stage training method achieve good performance. Based on ALTER, we have also produced MTA-equipped language models for various domains

Facilitating Self-monitored Physical Rehabilitation with Virtual Reality and Haptic feedback

Physical rehabilitation is essential to recovery from joint replacement operations. As a representation, total knee arthroplasty (TKA) requires patients to conduct intensive physical exercises to regain the knee's range of motion and muscle strength. However, current joint replacement physical rehabilitation methods rely highly on therapists for supervision, and existing computer-assisted systems lack consideration for enabling self-monitoring, making at-home physical rehabilitation difficult. In this paper, we investigated design recommendations that would enable self-monitored rehabilitation through clinical observations and focus group interviews with doctors and therapists. With this knowledge, we further explored Virtual Reality(VR)-based visual presentation and supplemental haptic motion guidance features in our implementation VReHab, a self-monitored and multimodal physical rehabilitation system with VR and vibrotactile and pneumatic feedback in a TKA rehabilitation context. We found that the third point of view real-time reconstructed motion on a virtual avatar overlaid with the target pose effectively provides motion awareness and guidance while haptic feedback helps enhance users' motion accuracy and stability. Finally, we implemented \systemname to facilitate self-monitored post-operative exercises and validated its effectiveness through a clinical study with 10 patients

MRCN: A Novel Modality Restitution and Compensation Network for Visible-Infrared Person Re-identification

Visible-infrared person re-identification (VI-ReID), which aims to search identities across different spectra, is a challenging task due to large cross-modality discrepancy between visible and infrared images. The key to reduce the discrepancy is to filter out identity-irrelevant interference and effectively learn modality-invariant person representations. In this paper, we propose a novel Modality Restitution and Compensation Network (MRCN) to narrow the gap between the two modalities. Specifically, we first reduce the modality discrepancy by using two Instance Normalization (IN) layers. Next, to reduce the influence of IN layers on removing discriminative information and to reduce modality differences, we propose a Modality Restitution Module (MRM) and a Modality Compensation Module (MCM) to respectively distill modality-irrelevant and modality-relevant features from the removed information. Then, the modality-irrelevant features are used to restitute to the normalized visible and infrared features, while the modality-relevant features are used to compensate for the features of the other modality. Furthermore, to better disentangle the modality-relevant features and the modality-irrelevant features, we propose a novel Center-Quadruplet Causal (CQC) loss to encourage the network to effectively learn the modality-relevant features and the modality-irrelevant features. Extensive experiments are conducted to validate the superiority of our method on the challenging SYSU-MM01 and RegDB datasets. More remarkably, our method achieves 95.1% in terms of Rank-1 and 89.2% in terms of mAP on the RegDB dataset

Dynamic Response of Slope Inertia-Based Timoshenko Beam under a Moving Load

In this paper, the dynamic response of a simply supported beam subjected to a moving load is reinvestigated. Based on a new beam theory, slope inertia-based Timoshenko (SIBT), the governing equations of motion of the beam are derived. An analytical solution is presented by using a coupled Fourier and Laplace&ndash;Carson integral transformation method. The finite element solution is also developed and compared with the analytical solution. Then, a comparative study of three beam models based on the SIBT, Euler&ndash;Bernoulli and Timoshenko, subjected to a moving load, is presented. The results show that for slender beams, the dynamic responses calculated by the three theories have marginal differences. However, as the ratio of the cross-sectional size to beam length increases, the dynamic magnification factors for the mid-span displacement obtained by the SIBT and Timoshenko beams become larger than those obtained by the Euler&ndash;Bernoulli beams. Furthermore, until the ratio is greater than 1/3, the difference between the calculated results of the SIBT and Timoshenko beams becomes apparent

Label-Free miRNA-21 Analysis Based on Strand Displacement and Terminal Deoxynucleotidyl Transferase-Assisted Amplification Strategy

MicroRNAs (miRNAs) are regarded as a rising star in the biomedical industry. By monitoring slight increases in miRNA-21 levels, the possibilities of multi-type malignancy can be evaluated more precisely and earlier. However, the inconvenience and insensitivity of traditional methods for detecting miRNA-21 levels remains challenging. In this study, a partially complementary cDNA probe was designed to detect miRNA-21 with target-triggered dual amplification based on strand displacement amplification (SDA) and terminal deoxynucleotidyl transferase (TdT)-assisted amplification. In this system, the presence of miRNA-21 can hybridize with template DNA to initiate SDA, generating a large number of trigger molecules. With the assistance of TdT and dGTP, the released trigger DNA with 3&prime;-OH terminal can be elongated to a superlong poly(guanine) sequence, and a notable fluorescence signal was observed in the presence of thioflavin T. By means of dual amplification strategy, the sensing platform showed a good response tomiRNA-21 with a detection limit of 1.7 pM (S/N = 3). Moreover, the specificity of this method was verified using a set of miRNA with sequence homologous to miRNA-21. In order to further explore its practical application capabilities, the expression of miRNA in different cell lines was quantitatively analyzed and compared with the qRT-PCR. The considerable results of this study suggest great potential for the application of the proposed approach in clinical diagnosis

3D Digital Adaptive Thorax Modelling of Peoples with Spinal Disabilities: Applications for Performance Clothing Design

Peoples with spinal disability face a huge problem in the design and development of ergonomically fitted and comfortable clothing. Various research studies on the design and developments of functional clothing for scoliosis patients consider their morphological shapes. However, developing appropriate models of the complicated and deformed anatomical shape of the patient in 3D digitization technologies makes it possible to design a comfortable and fitted garment. The current paper proposes a method for developing a fully parametric 3D adaptive model of the thorax of a patient suffering from scoliosis. The model is designed from the spine and follows the deformation of the spine to adapt the thorax skeleton according to the temporal evolution of the spinal column deformation. The integration of the model of the thorax, adjusted to the patient’s data, enables the chain of acquisition, processing, and global model to be validated. The fit of the model could be improved for the different bones and it is possible to modify the angles of the spine to see the evolution of the disease. The developed model greatly helps to further detect anthropometric points from certain bone parts of the skeleton to design a basic bodice adapted to the patient’s evolving morphology

Copper-Catalyzed Synthesis of Indol-3-yl α‑(Difluoromethyl)-α-(trifluoromethyl)carbinols: Construction of Difluoromethylated sp³ Carbon Centers

An efficient copper-catalyzed synthesis of indol-3-yl α-(difluoromethyl)-α-(trifluoromethyl)­carbinols is developed. The reaction proceeds in good to excellent yields through a Friedel–Crafts-type mechanism, and a variety of indoles with commonly occurring functional groups such as formyl, cyano, nitro, alkyloxide, and halogen are well tolerated. In addition, these carbinol products are readily transformed into diversified difluoromethylated dinitrile indol-3-yl derivatives. This strategy provides a general synthetic method for ready construction of difluoromethylated sp³ carbon centers

In Situ Polymerization to Boron Nitride-Fluorinated Poly Methacrylate Composites as Thin but Robust Anti-Corrosion Coatings

High-performance anti-corrosion coatings featuring easy processability and thin thickness are highly desired in industry. Yet, solution process coating often faces a sedimentation problem with particles which are used as reinforcement in coatings. In this contribution, boron nitride (BN) was modified by an acrylate silane coupling agent (KH-570) to obtain acrylated BN flakes. Afterwards, the acrylated BN flakes were in situ copolymerized with 2-(perfluorohexyl)ethyl methacrylate to synthesize BN-fluorinated poly methacrylate (PFBP) composites. The as-obtained PFBP composites can form stable coating solutions, in which sedimentation of BN flakes seldom happens. The coating solution can easily form uniform coatings on various substrates with nanoscale thickness, confirmed by scanning electron microscope (SEM). The corrosion resistance of the samples coated PFBP coatings in 3.5 wt.% sodium chloride solution was evaluated by electrochemical impedance spectroscopy (EIS). It is indicated that the incorporation of BN flakes greatly reduce the corrosion rate. Adhesion measurements and abrasion resistance test indicate the PFBP coating performs good adhesion to substrate and robustness. Through the in situ polymerization, acrylated BN flakes are connected with the polymer chain, which inhibits the sedimentation of BN in the coating solution. Additionally, the BN flakes dispersed in the fluorinated polymer act as barriers, improving the corrosion resistance of the coated samples